The performance of Machine Learning (ML) and Deep Learning (DL)-based Intrusion Detection and Prevention Systems (IDS/IPS) is critically dependent on the relevance and quality of the datasets used for training and evaluation. However, current AI model evaluation practices for developing IDS/IPS focus predominantly on accuracy metrics, often overlooking whether datasets represent industry-specific threats. To address this gap, we introduce a novel multi-dimensional framework that integrates the MITRE ATT&CK knowledge base for threat intelligence and employs five complementary metrics that together provide a comprehensive assessment of dataset suitability. Methodologically, this framework combines threat intelligence, natural language processing, and quantitative analysis to assess the suitability of datasets for specific industry contexts. Applying this framework to nine publicly available IDS/IPS datasets reveals significant gaps in threat coverage, particularly in the healthcare, energy, and financial sectors. In particular, recent datasets (e.g., CIC-IoMT, CIC-UNSW-NB15) align better with sector-specific threats, whereas others, like CICIoV-24, underperform despite their recency. Our findings provide a standardized, interpretable approach for selecting datasets aligned with sector-specific operational requirements, ultimately enhancing the real-world effectiveness of AI-driven IDS/IPS deployments. The efficiency and practicality of the framework are validated through deployment in a real-world case study, underscoring its capacity to inform dataset selection and enhance the effectiveness of AI-driven IDS/IPS in operational environments.

Machine Learning also provides a great tool to conduct predictive analysis to understand risk and plan accordingly, and in some cases provides …

Many governments worldwide are looking at using Artificial Intelligence (AI) and other cognitive technologies as part of making their operations more efficient, better serving their citizens, and increasing the range of ways they can meet their missions. It's no surprise then that the US Government and forward thinking leadership is making investments into AI technologies. Additionally, some agencies such as the US Department of Health & Human Services (HHS) are also seeing how other emerging technologies such as blockchain can help. Jose Arrieta, the CIO at the US Department of Health & Human Services (HHS), interviewed on a recent AI Today podcast episode while he was the associate Deputy Assistant Secretary for Acquisition at HHS is one such leader who sees how AI and blockchain can have a big impact at the agency. He became a program manager to build IT systems to use machine learning to analyze biographical information about people.